Graph Network based Approaches for Multi-modal Movie Recommendation System

The three times increase of SonyLiv viewers during the Tokyo Olympic, the 10% hike of YouTube users during the isolation era of covid-pandemic, and the 19% growth in Netflix user count due to the fastest growth of OTT, etc. have made the digital platform's mode all-time active and specific. The hourly increase of users' interactions and the e-commerce platform's desire of letting users engage on their sites are pushing researchers to shape the virtual digital web as user specific and revenue-oriented. This paper develops a deep learning-based approach for building a movie recommendation system with three main aspects: (a) using a knowledge graph to embed text and meta information of movies, (b) using multi-modal information of movies like audio, visual frames, text summary, meta data information to generate movie/user representations without directly using rating information;this multi-modal representation can help in coping up with cold-start problem of recommendation system (c) a graph attention network based approach for developing regression system. For meta encoding, we have built knowledge graph from the meta information of the movies directly. For movie-summary embedding, we extracted nouns, verbs, and object to build a knowledge graph with head-relation-tail relationships. A deep neural network, as well as Graph attention networks, are utilized for measuring performance in terms of RMSE score. The proposed system is tested on an extended MovieLens-100K data-set having multi-modal information. Experimental results establish that only rating-based embeddings in the current setup outperform the state-of-the-art techniques but usage of multi-modal information in embedding generation performs better than its single-modal counterparts. 1. © 2022 IEEE.

Ligand Binding Prediction Using Protein Structure Graphs and Residual Graph Attention Networks.

Computational prediction of ligand-target interactions is a crucial part of modern drug discovery as it helps to bypass high costs and labor demands of in vitro and in vivo screening. As the wealth of bioactivity data accumulates, it provides opportunities for the development of deep learning (DL) models with increasing predictive powers. Conventionally, such models were either limited to the use of very simplified representations of proteins or ineffective voxelization of their 3D structures. Herein, we present the development of the PSG-BAR (Protein Structure Graph-Binding Affinity Regression) approach that utilizes 3D structural information of the proteins along with 2D graph representations of ligands. The method also introduces attention scores to selectively weight protein regions that are most important for ligand binding. Results: The developed approach demonstrates the state-of-the-art performance on several binding affinity benchmarking datasets. The attention-based pooling of protein graphs enables identification of surface residues as critical residues for protein-ligand binding. Finally, we validate our model predictions against an experimental assay on a viral main protease (Mpro)-the hallmark target of SARS-CoV-2 coronavirus.

Finding Asymptomatic Spreaders in a COVID-19 Transmission Network by Graph Attention Networks.

In the COVID-19 epidemic the mildly symptomatic and asymptomatic infections generate a substantial portion of virus spread; these undetected individuals make it difficult to assess the effectiveness of preventive measures as most epidemic prevention strategies are based on the detected data. Effectively identifying the undetected infections in local transmission will be of great help in COVID-19 control. In this work, we propose an RNA virus transmission network representation model based on graph attention networks (RVTR); this model is constructed using the principle of natural language processing to learn the information of gene sequence and using a graph attention network to catch the topological character of COVID-19 transmission networks. Since SARS-CoV-2 will mutate when it spreads, our approach makes use of graph context loss function, which can reflect that the genetic sequence of infections with close spreading relation will be more similar than those with a long distance, to train our model. Our approach shows its ability to find asymptomatic spreaders both on simulated and real COVID-19 datasets and performs better when compared with other network representation and feature extraction methods.

Fine-grained Detection of Academic Emotions with Spatial Temporal Graph Attention Networks using Facial Landmarks

With the incidence of the Covid-19 pandemic, institutions have adopted online learning as the main lesson delivery channel. A common criticism of online learning is that sensing of learners' affective states such as engagement is lacking which degrades the quality of teaching. In this study, we propose automatic sensing of learners' affective states in an online setting with web cameras capturing their facial landmarks and head poses. We postulate that the sparsely connected facial landmarks can be modelled using a Graph Neural Network. Using the publicly available in the wild DAiSEE dataset, we modelled both the spatial and temporal dimensions of the facial videos with a deep learning architecture consisting of Graph Attention Networks and Gated Recurrent Units. The ablation study confirmed that the differencing of consecutive frames of facial landmarks and the addition of head poses enhance the detection performance. The results further demonstrated that the model performed well in comparison with other models and more importantly, is suited for implementation on mobile devices with its low computational requirements.

The Experience with the REU-sponsored Project on Predicting COVID-19 Pandemics Using Physics-Guided Graph Attention Networks

The COVID-19 pandemic has significantly impacted most countries in the world. Analyzing COVID-19 data from these countries together is a prominent challenge. Under the sponsorship of NSF REU, this paper describes our experience with a ten-week project that aims to guide a REU scholar to develop a physics-guided graph attention network to predict the global COVID-19 Pandemics. We mainly presented the preparation, implementation, and dissemination of the addressed project. The COVID-19 situation in a country could be dramatically different from that of others, which suggests that COVID-19 pandemic data are generated based on different mechanisms, making COVID-19 data in different countries follow different probability distributions. Learning more than one hundred underlying probability distributions for countries in the world from large scale COVID-19 data is beyond a single machine learning model. To address this challenge, we proposed two team-learning frameworks for predicting the COVID-19 pandemic trends: peer learning and layered ensemble learning framework. This addressed framework assigns an adaptive physics-guided graph attention network (GAT) to each learning agent. All the learning agents are fabricated in a hierarchical architecture, which enables agents to collaborate with each other in peer-to-peer and cross-layer way. This layered architecture shares the burden of large-scale data processing on machine learning models of all units. Experiments are run to verify the effectiveness of our approaches. The results indicate the proposed ensemble outperforms baseline methods. Besides documented on GitHub, this work has resulted in two journal papers. © 2022 IMCIC 2022 - 13th International Multi-Conference on Complexity, Informatics and Cybernetics, Proceedings. All rights reserved.

STAN: spatio-temporal attention network for pandemic prediction using real-world evidence.

OBJECTIVE: We aim to develop a hybrid model for earlier and more accurate predictions for the number of infected cases in pandemics by (1) using patients' claims data from different counties and states that capture local disease status and medical resource utilization; (2) utilizing demographic similarity and geographical proximity between locations; and (3) integrating pandemic transmission dynamics into a deep learning model. MATERIALS AND METHODS: We proposed a spatio-temporal attention network (STAN) for pandemic prediction. It uses a graph attention network to capture spatio-temporal trends of disease dynamics and to predict the number of cases for a fixed number of days into the future. We also designed a dynamics-based loss term for enhancing long-term predictions. STAN was tested using both real-world patient claims data and COVID-19 statistics over time across US counties. RESULTS: STAN outperforms traditional epidemiological models such as susceptible-infectious-recovered (SIR), susceptible-exposed-infectious-recovered (SEIR), and deep learning models on both long-term and short-term predictions, achieving up to 87% reduction in mean squared error compared to the best baseline prediction model. CONCLUSIONS: By combining information from real-world claims data and disease case counts data, STAN can better predict disease status and medical resource utilization.